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'0 Method for detecting change points In motion picture Images. 

" - 

0 There is provided a method for detecting change points between cuts from motion picture images including 
a^pluralitv of consecutive images^irst o f all. in this method, motion picture images which are the subject of 
deWtion\ of change betweeh^cute are inputted ^21^ In a time series bv takinp a frame as ttie unit^l andla 
predetermined feature quantity including a color histogram possessed by image data of said motion (MCture 
images/is produced (22) while taking a frame as the unit. A correlation coefficient with respect to feature quantity 
between the above described feature quantity and a feature quantity produced in an immediately preceding 
franrte is then produced (23). Further, a change rate between the conelation coefficient of the current frame and 
'-^aconrelation coefficient produced for a frame preceding the cun-ent frame is produced (24). A time point whereat 
the change rate exceeds a predetenmined alk>wable value is detected (25) as a change point between cuts of the 
motion picture images. 
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BACKGROUND OF THE INVENTION 

The present Invention relates to a motion picture image editing method capable of searching for the 
starting point of motion picture images included in videotapes and/or moviefilm signals separately for 
s individual scenes (hereafter referred to as cuts), and in particular to a picture image change point detecting 
method for detecting a change point between cuts from motion picture images stored in video tape or a 
video disk. 

As a treatise concerning a method for automatically detecting motion picture image cuts. "Study on 
motion picture image data base handling" by Sotomura and Abe» Technical Report tE89-6. 89-33. The 
10 Institute of Electronics, Information Communication Engineers (lEICE), for example, can be mentioned. This 
treatise has proposed a meti^ of detecting pictoriol image change points visually by defining "motion 
picture image cut" as a section of picture images in which the picture images largely change and by using 
brightness. That is to say. histogram of brightness is extracted as a feature quantity for each frame of a 
subject motion picture image, and then a histogram difference between a ^ame at the pertinent current time 
16 point and its immediately preceding frame is derived. The sum total of resultant difference values is 
calculated. The sum total of difference values is used as the frame correlation coefficient, and a certain 
fixed threshold is set. When the correlation coefficient exceeds this threshold value, it is determined that a 
change point beNyeen cuts has appeared. A "cut" means a section of picture images forming one scene 
lasting from the beginning of a certain operation to the end thereof in motion* picture images such as video 
20 signals and picture image signals. 

SUMMARY OF THE INVENTION 

However, the above described method uses tiie difference of brightness histogram of tiie whole screen 
25 of each frame as the conelation coefficient If the difference of brightness histogram is thus used to detect 
picture image change points, detection sensitivity is insufficient. This will now be described. It is now 
assumed that tiie conrelation coefficient has a^ali value , althou gh the picture image changes completely. 
For example, the difference value of brightness histogram characteristic t>etween a picture having a white 
left half and a blacic right half as shown in FHg. ISA and a picture having a black left half and a white right 
30 half as shown in Rg. 15B becomes 0 in this case as evident from characteristics shown in brightness 
histogram characteristic diagrams Pigs. 16A and 166 respectively corresponding to Rgs. ISA and 158. 
Therefore, the correlation coefficient also becomes 0. Even if there are changes in individual places of a 
picture, it is determined that there is little change in the histogram provided that changes of their total 
values are the same. 

3s FrorrnRirs^facts. it is considered that the detection sensitivity has been deteriorated as a result of 
mutual interference between brightness feature quantities of picture image areas included in one frame. 

Further, in case this conrelation coefficient is used for detecting cuts of picture images moving violently, 
individual correlation coefficients assume Jarge^ ya^!ues_in_ suc cession every frame_o LDicture imao es^and ^ 
hence false detection j)f_mpti(yi, picnjre, i^^^ in contrast witii the case described 
40 before. In picture images obtained by zooming-in a sports player moving violently, for example, picture 
image changes continue constantly. Therefore, a change po int_, between c u ts occurs ever y frame, conrect 
cut detection becoming difficult, ......^,1^™.-.-..!- , 

A change point t)etween cuts always present even between frames having rapidly changing 

picture images. Hence, it follows that^if Is a necessary condition but not sufficient condition to detect a 
45 frame having a rapidly changing picture image as compared with receding and succeeding frames. 

An object of tiie present invention is to provide a motion picture image change point detecting method 
capable of settling the above descrit^ed prok>lems of the prior art. preventing false detection in case of 
consecutive frames having violent motions, and detecting at high speed conrect change points of motion 
picture images even in an ordinary work station witiiout lowering the detection sensitivity. 
50 In order to achieve the above descrit>ed object, a motion picture image change point detecting method . t ^^^^ 

according to ttie present invention has a feature (i) ttiat this method comprises the steos of Inputting motion. " \ ^ y<P ' Q 
j)icture image s to be processe d to a pr^ essing unit in ^ ^ a^time^serjes.bv takin g a frame as the un it. S-'^^^^^'^'^T^^ 
, Y/'^ ^MLaqng^g^ggggjfitr^ by that image dat a,by^taklng a frame ^ ^ 

\^ as *I)Q w i t deriving a correlation^'coefficleht between that feature quantity and a feature Quantitv^calcula^y l / \ \ 

^ l56 Ib-'kW "immediately pre<5eaing frame, defivifTg^FcKange rate between tiiat correlation coefficient and a aA^/ 

\ Xd correlation coefficient derived in a past frame, and determining the time point whereat J he change rate ^ 
ijfijW N. ^jggggdgja eredetermined aUpi^dble value as a change point between cuts of motion ^picture Ima'ges.' ' 
urtner. (ii)'the motion picture image changel)o>nt detecting method according to the presc 
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a feature that the feature quantity includes a brightness or color histogram of the case where an Image is 
divided into small blocks. Further, (ili) the motion picture Image change point detecting method according to 
the present invention has a feature jha^ feature Quanl itle s of coTOSPondin^ blocks of consecutive preceding 
and succeeding frames in case ^ure1rnages"are SiviciedHHffi'smairb^ compared. bk>cks largely 

5 changed as a result of comparison are counted, and the number of counted blocks normalized by the 
number of blocks of the vifhole image Is defined as the correlation coefficient. Further, (iv) the motion picture 
image change point detecting method according to the present invention has a feature that the change rate 
is defined as the difference between the correlation value derived in the current frame and a larger one of 
the corretatk>n value derived in a frame immediately preceding the current frame and the conflation value 

10 derived in a frame preceding me cunrent frame by two frames. Further, (v) the motion picture image change 
point detecting method according to the present Invention has a feature that motion picture images to be 
processed are inputted by taking a frame as the unit and upon determination of a change point t)etween 
cuts of motion picture images, a time code identifying the determined frame and a picture image of a 
determined change poirvt between cuts are outputted as index information of that motion picture image cut. 

75 In accordance with the present invention, motion picture images are inputted to a processing unit in 
time series by taking a frame as the unit, a feature quantity representing the feature of picture image data is 
calculated every section area (herein referred to as block) included in each frame, and the degree of 
resemblance between this feature quantity and a feature quantity calculated in an immediately preceding 
frame is derived every block. The ratio of the number of blocks each having a low degree of re$emt>lance 

20 to the number of all blocks is derived as a correlation coefficient A change rate between this conrelation 
coefficient and a correlation coefficient derived in a past frame is calculated. . When this change rate 
exceeds an allowable value, it is determined that a change point of the motion picture image has appeared. 
A time code identifying the frame of that time point and a picture image of a change point k>etween cuts are 
outputted as index information of the motion picture image cut. 

25 In this way. a change Is detected for each of individual blocks included in a frame, and the number of 
changed blocks normalized by the number of all blocks is used as the conrelation coefftolent for detecting a 
change point between motion picture image cuts. Therefore, interference of feature quantities between 
areas can be prevented. In addition, since the correlation coefficient is normalized in the range of 0 to 1, 
handling in the succeeding processing becomes easy. 

30 Further, in case a change point between motion picture image cuts is to be detected, determination is 
made on the basis of a change rate with respect to a past correlation coefficient. Even if the value of 
correlation coefficient is large (i.e.. even if the picture image change is large), therefore, the change rate 
itself is small provided that it continues from the past. Therefore, it is prevented that this is falsely regarded 
as a cut and detected. That is to say. such a state that a picture image change is large but its change rate 

35 is small continues in cuts having violent motions. Therefore, use of the above described method is 
extremely effective. 

BRIEF DESCRIPTION OF THE DRAWINGS 

40 Rg. 1 is a general block diagram of an automatic indexing system of a motion picture image which is an 
embodiment implementing the present invention method; 

Fig. 2 is a PAD diagram of index information producing processing of motion picture image cuts in the 
apparatus of Rg. 1; 

Rg. 3 is a PAD diagram showing details of initialization processing of Fig. 2; 
45 Rg. 4 is a PAD diagram showing the detail procedure of picture image decimation processing of the 
same Rg. 2; 

Fig. 5 is a diagram showing the concept of picture image decimation processing of Fig. 4; 
Rg. 6 is a diagram showing an example of cotor bit compression in Rg. 5: 

Rg. 7 is a PAD diagram showing the detailed procedure of correlation coefficient calculation in the same 
50 Rg. 2: 

Figs. 8A and 8B are diagrams showing the principle of histogram calculation in one divided frame picture 
image of the present invention; 

Rgs. 9A and 9B are diagrams for explaining the principle of an Interframe correlation coefficient 
according to the present Invention; 
55 Rg. 10 is a PAD diagram showing details of a correlation coefficient change rate calculating method of 
the same Rg. 2; 

. Rg. 11 is a PAD diagram showing details of index information producing processing of the same Rg. 2; 
Rg. 12 is a diagram showing the structure of various arrangement data in a main memory included in the 
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configuration of Rg. i; 

Fig. 13 is a PAD diagram showing details of frame update processing included In Rg. 2: 
Figs. 14a and 14B are time charts respectively showing the change of correlation coefficient and 
correlation coefficient change rate with time obtained by the present invention are applied; and 
s Figs. ISA and 15B and Figs. 16A and 18B respectively show examples of images having left and right 
halves of the screen as bright and dark portions* and characteristic diagrams in case where brightness 
histograms are obtained by taking one franne as the unit. 

OESCniPTION OF THE PREFERRED EMBODIMENT 

Embodiments of the present Invention will hereafter be described in detail by referring the drawings. 
Fig. I IS a general block diagram of an automatic motion picture image indexing system showing an 
embodiment for tmplomonting the present invention method. 

With rofoionco to Fig. 1. numerals 10. 11 and 12 denote input/output devices of motion picture images 

16 to be procossod. And numorais 10, 11 and 12 denote a video disk device, a video tape recorder and a 
television tuner, respectively. Further, numeral 13 denotes a video controller for performing control of 
motion picture image input output devices and changing inputs. Numeral 14 denotes a frame memory for 
storing motion picture signals frame by frame. 15 a central processing unit for issuing directions to the 
video contiollor 13 to exercise control. 16 a main memory for storing program to be executed by the central 

20 processing unit 15 and data to be used, and 17 a magnetic disk device which is an external memory for 
supplementing the capacity of the main memory 16. 

Operational features of motion picture Image input/output devices 10, 11 and 12 will now be described. 
In the video disk device 10. video data are sequentially recorded, but random access is possible. In the 
video tape recorder 11. video data are sequentially recorded and access is also sequential, but stow 

25 reproduction is possible. Further, in the television tuner 12, recording is impossible and images are 
transmitted (broadcasted) unilaterally from a broadcasting station or the like at a constant speed. 

On the basis of a command 18 supplied from the central processing unit 15. the video controller 13 
exercises control over various motion pteture image input/output devices 10, 11 and 12, and connects one 
of them as the input source to take in motion picture Images and store the motion picture images frame by 

30 frame. The central processing unit 15 analyzes motion picture images read out from the frame memory 14, 
and stores intermediate data thus analyzed into the main memory 16 temporarily. Thereafter, the central 
processing unit 15 produces a time code identifying a frame at that time point and an Image of a change 
point between cuts as index information of motion picture images and stores them into the magnetic disk 
device 17. When occasion demands, the central processing unit 15 can also output a write command signal 

35 and write a time code indicating a change point between cuts into the video disk devk» 10 and the video 
tape recorder 11. 

Thus, a system of the present invention does not include special configuration and it can be easily 
constructed by using AV devices and work station devices. 

Fig. 2 is a PAD diagram of index infonmatton producing processing performed by the central processing 
40 unitofRg. 1. 

The central processing unit 15 detects image change points of motion picture image cuts and thereby 
produces index information. 

Rrst of all, initial values are supplied to various variables in initialization processing (box 20). Details of 
the initialization processing will be described later by refening to Rg. 3. 
45 In image input processing (box 21), motion picture Images are taken Into the frame memory 14 by one 
frame, ff in this case the input source is the vkJeo disk device 10 or the video tape recorder 11. information 
conresponding to an address where that h^ame is stored on a medium is read as the time code and stored 
into a variable TM. 

In image preprocessing (box 22), image decimation and color picture element compression are 
50 performed in order to reduce the amount of computation of succeeding processing. Details of image 
decimation processing wilt be described later by referring to Rgs. 4 and 5. 

In frame congelation coefficient calculation processing (box 23), the frame correlation coefficient is 
calculated and the result of calculation is stored into arrangement 0(t). 

As the frame con-elation coefficient used In the present Inventkm. the proportion that one frame of a 
S5 changed area included in a corresponding area between frames accounts for in the whole screen is used. 
By doing so. the frame correlation coefficient ranges from 0 to 1. If the image completely changes between 
frames, the frame conrelation coefficient becomes 1. If there is no change, the frame correlation coefficient 
becomes 0. There are various methods for deriving a change area. Simple difference of picture elements 
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results in high sensitivity to noises. Further, in birghtness histogram difference of the whole screen of each 
frame, the detection sensitivity is worsened by interference of brightness twtween areas as shown in Figs. 
ISA and 15B and Rgs. 16A and 16B. In the present embodiment, therefore, the screen is divided into snnall 
blocks, a change block in the current frame Is derived on the basis of the degree of resemblance of cotor 

5 histogram between corresponding blocks of frames, and the sum total of these change blocks is interpreted 
as a change area, concrete conrelation coefficient calculation will be described later by refemng to Fig. 7. 

In succeeding change rate calculation processing (box 24). the change rate of the correlation coefficient 
is calculated, and then the result of calculation is stored into a variable DDT. A concrete calculation method 
of the change rate of the correlation coefficient will be described later in detail by referring to Fig. 10. 

to In the succeeding decision processing (box 25). a change point between cuts is determined. 
That is to say, it is determined that a change between cuts has appeared when 



f% dD(t) 

dt 



Here. ^ is a threshold for deciding the change point between cuts. In the present embodiment, fi = 0.6. 
n - In image change point registration processing (box 26), index Information of an operation Image cut is 
pffwiiutfvl in case it is determined that a change point t)etween cuts has appeared. Details of index 
inlnfmation producing processing of the operation image cut will be described later by referring to Rgs. 11 
and 12. 

In frame update processing (box 27). the frame is updated. Details of this processing will be described 
25 later by referring to Rg. 13. 

In an video frames decision processing (box 28). it is determined whether there is still a motion picture 
image to t>e processed or not. If there is still a motion ^cture image to be processed, processing is 
repeated from block 21. 

Fig. 3 is a detailed flow chart of initialization processing (box 20) of Rg. 2. 
30 The initial value of N for storing the number of image change points is set at 0 (box 20-1). 

Further, variables t. t*l. t*2 are used as subscripts for address reference of various anrangements 
respectively indicating the current frame, a frame preceding the current frame by one frame, and a frame 
preceding the current frame by two frames. 

As initial values. 0. 1 and 2 are supplied respectively to 1. 1-1 and t-2 (box 20*2). 
35 Succeedingly. correlation coefficients D(t-1) and D(t-2) respectively of the frame preceding the current 
frame by one frame and the frame preceding the current frame by two frames are provided with 0 (box 20- 
3). 

Rg. 5 is a diagram showing the state of image decimation and compression of color picture element 
values in the frame memory of Rg. 1. Rg. 6 is a sut>sidiary explanation diagram for Rg. 5. 

40 With reference to Rg. 5. numeral 14-1 denotes an input image P, 14-2 a color image Q after picture 
element decimation, and 14-3 a color code image F after complession of color identification bits. 
Longitudinal and lateral (^cture elements of the input image 14-1 are decimated respectively to 1/8, and 
color identification bits compressed respectively to 1/4. For convenience, reduction of picture elements and 
reduction of cotor bits are hereafter thus referred to as decimation and compression, respectively. 

45 The color code image 14-3 after compression has information compressed to 1/256 as compared with 
the input image. Even if succeeding Image processing is performed with respect to this compressed image, 
image change points can be detected nnore frequently than several frames per second even at an ordinary 
work station t)ecause of reduced information content 

As for compression of color bits, only two leading bits of 8 bits of each of B, G and R are taken out to 

50 form compressed bits as shown in Fig. 6. 

Rg. 4 is a PAD diagram showing image decimation processing and cotor picture element value 
compression processing of Fig. 2. In Rg. 4. box 22-1 shows a concrete example of image decimation 
processing, and box 22-2 shows a concrete example of compression processing of color picture element 
values. 

55 Rrst of ail, image decimation is performed by the following equations, 
(i) Image decimation 
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Qr(t, X, y) = Pr(t, m.x, m.y) 
Qg(t. X, y) = Pg(t. m*x, m.y) 
Qb(t. X. y) = Pb(t, m.x, m.y) (1) 

5 Here. P denotes an input color image, Q a color image after decimation (referred to as compressed 
image), t a frame number, m a decimation value, and • a multiplication symbol. Further, subscripts r. g and 
b represent components of three primary colors. In standard dynamic picture images, the image size is 640 
x 480 picture elements. Here, m is a compression value. To be concrete, m is &: The larger this value is, 
the larger the degree of compression becomes. When the value of m is large, however, there is also a 

10 possibility of lowering in detection performance. Therefore, the value of 8 has been determined by 
experimentally confirming that this value brings about good performance. Thereby, the amount of computa- 
tion is reduced to 1/64 in the succeeding processing. 

As for x and y addresses of images after compression in this case, x ranges from 1 to 80, whereas y 
ranges from 1 to 60. The frame number t of the above described equation (1) Is stored into the variable TM. 

15 Further, in the present embodiment, it is not necessary to store sequentially images in the frame memory 
14. What is needed for processing at the time point of processing of the frame number t is only one sheet 
of image at that time. 

Block 22-1 of Rg. 4 shows concrete decimation pnxessing. It Is shown that the color image 
(compressed image) after decimation has coordinates obtained by multiplying x and y of the input color 
20 image P by the decimation value m. 

(ii) Compression of color picture element values 

F(t. X. y) = Qr(t. x, y).f + Qg(t. x. y).f.R + Qb(t. x, y)^.FUR (2) 

25 

Here. F denotes a color code image after compression, f a compression coefficient of motion range of 
picture element values (f = 1/64 in the example of Rg. 5). and R a motion range of each color component 
after compression (R = 4 in the example of Rg. 5). 

Further, the motion range of the color code Image F is the cube of the motion range of R. Further, the 
30 compression coefficient f of the motion range is the same for respective components, t>ut it may be set 
individually. Standard motion picture images have 24-bit colors, but they are herein compressed to 6-bit 
colors. (As shown in Rg. 6, two leading bits are taken out of 8 bits representing each color picture element 
value.) 

In box 22-2 of Rg. 4, concrete compression processing of color picture element values is shown. Two 
J5 high-order bits of each component are cut out and combined with bits of other components, resulting in a 6- 
bit composite signal, that is to say. the term R in the color image Q after decimation processing is cut out 
by AND-Ing data shifted by 6*bits and a numerical value 3. The term Q is cut out by AND-ing data shifted 
by 12 bits and a numerical value 12. The term B is cut out by AND-ing data shifted by 18 bits and a 
numerical value 48. These terms thus cut out are OR-ed to be combined, resulting in 6 bits in total 
40 comprising three components each having 2 bits. 

Fig. 7 is a PAD diagram of frame con-elation coefficient calculation processing of Rg. 2. 
Concrete calculations of correlation coefficient are performed by the following equations. 

(i) Histogram calculation of each blockr 

45 

G(t, k, x, y) = F(t. X + AxJoc, y + Ay-ky) (3) 



Ax A y 

if (t, k, X, y) = i (4) 

55 

Here, Q denotes a block image. H a block histogram (i.e.. a histogram counter value). F a color code 
image after compression, k a block serial number, kx a block number in the laterial direction, ky a block 
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number in the longitudinal direction. Ax the number of picture elements In the latteral direction of the block. 
Ay tiie number of picture elements In the longitudinal direction of the block, and I a color code number. To 
be conaete, kx ranges from 0 to 7, and ky ranges from 0 to 5, whereas Ax = 10 and Ay = 10. The block 
serial number k ranges from 0 to 47. Further, i ranges from 0 to 63. The block serial number k, the bkxk 
5 number kx in the lateral direction, and the block number ky in the longitudinal directkKi are related by k » 
kx + 8.ky. 

Equation (3) means that x and y of the block number k at time t of the block image G are obtained by 
cutting out time t included in the color code image F after compression, x ♦ Ax . kx, and y + Ay . ky. 
Further, equation (4) means tiiat contents of the histogram counter H are updated with respect to tiie block 

10 size Ax and Ay provided that the brightness in the place (x. y) in the block k is i. That is to say, each 
picture element (Ax. Ay) of a plurality of divided bk>ck images Q is generally scanned as indicated by an 
arrow of Fig. 8B. If the color code is I. the value of the histogram counter H In that position is updated. That 
is to say. if the value is i in a graph having an axis of abscissas representing i and an axis of ordinates 
representing H as shown in Fig. 8A, the histogram counter value H in tiie place of i is updated. In ttiis way. 

IS it is possible to know the histogram of the color code i of each block with respect to ttte brightness. 

In Fig. 7, blocks 23-1, 23-2 and 23-3 represent calculation steps for deriving the histogram. In tiie block 
23-1. a histogram storage area HIST Is initialized to have 0. 

In tiie btock 23-2, indexing cateulation of the block No. k and extraction processing of the color code i 
are performed. On the basis of tiie result, the pertinent histogram is updated by 1 in ti)e block 23-3. 

20 Since this processing is performed for each picture element of tiie image F only once, calculations are 
performed 80 x 60 = 4800 times. Unless images are. decimated in the preprocessing, calculations must be 
perfomned 307,200 times which are equivalent to 84 times 4800. 



25 



30 



(ii) Calculation of degree of histogram resemblance between bk)cks 
V(t. k. i) = (H(t. k. i) - H (t-1, k. i)yH(t-1 . k. i) (5) 



1 Nc-1 5 

k) Z V^(t, k, i) (6) 

i=0 



Here, V denotes a normalized difference value of histogram. E a degree of histogram resemblance, and 
dS Nc a color code number. That is to say, V is a difference value obtained by individually normalizing 
respective elements of the histogram. Thereby, difference values of respective color codes have equal 
weights irrespective of tiie distribution shape of the histogram. Further, E is ttie mean square of V and E 
does not depend upon whettter tiie color code number No is large or small. This means that the value of E 
does not change essentially even if the image compression rate is changed suitably to a different value in 
40 preprocessing. 

The color code number Nc has been reduced to 64 by preprocessing. In tiie above described equation 
(5) of the present embodiment i is changed from 0 to 63 and k is changed from 0 to 47. In order to 
calculate for all cases, calculations must be made 3072 times in total, in tiie above described equation (6). k 
is changed only from 0 to 47, and hence calculations are made 48 times. Since i is changed from 0 to 63 in 
45 the equation (6). therefore, the processing time substantially becomes a value proportionate to 3072 times 
in the same way as the equation (5). If images are not compressed in the preprocessing, calculations are 
performed 2^»= 12,582,912 times in equation (5) and (6). Further, in the present embodiment, ttie 
histogram H need only store histograms corresponding to two frames at most of the time t and time t-1. 
Therefore, tiie necessary capacity of ttie main memory 16 is reduced. 

so 

Oil) Calculation of Interframe correlation coefficient 



S5 



. Nk-1 
D(t) = ^5!r 

k=0 



if £(t, k) 2 a 



(7) 
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^ Here, D denotes a correlation coefficient, and i: denotes the number of blocks which have changed. 
Since D Is derived by dividing the number of blocks which have changed by the total number Nk of blocks, 
the proportion of an area which has changed with respect to the whole screen Is approximately derived. 
Therefore, the precision can be improved by increasing Nk. Herein, Nk = 48. Further, the parameter a is a 
threshold for determining whether a block has changed or not Empirically, a is defined as a = 0.5. 

This is illustrated in Fig. 9A. For each of divided blocks, a change between a frame t and a frame t-1 
preceding the frame t by one frame is checked. A change produces 1 , whereas no change produces 0. In 
this way. the number of blocks which have changed is divided by the number of overall btocks. The same 
processing is then performed between the frame t-1 and a frame t-2 preceding the frame t-1 by one frame. 
In this way, changes of respective blocks are successively checked. Succeed! ngly, as shown in Fig. 9B, 
instead of the change of frame correlation coefficient as compared with that of the frame t-1 preceding the 
current frame by on© frame, changes of frame correlation coefficients of t-1 and the frame t-2 preceding the 
current frame by two frames are compared, and the difference between the frame correlation coefficient and 
a lorgor change obtained by the above described comparison is derived, and the change rate of the frame 
correlation coefficient is obtained. Thereby, it is possible to prevent dispersion among frames. 

Blocks 23-4 to 23-11 shown in Fig. 7 are calculation steps for deriving the correlation coefficient D(t). 
First of all, the conrelation coefffcient D(t) is set at 0 in the block 23-4, The degree E of histogram 
resemblance of small blocks is derived In bk>cks 23-5, 23-8 and 23-7 and divided for respective cotor codes 
in block 23-8. When a fixed threshold is exceeded in decision block 23-9, the conrelation coefficient D(t) is 
updated by one in block 23-10. The processing heretofore descrih^ed is performed with respect to all of the 
small hinrks. The resultant correlation coefficient D(t) is divided by the number of all blocks in block 23-1 1 . 
to dnrivo the final correlation coefficient D<t). 

Fig. 10 is a flow chart of correlation coefficient change rate calculation processing. 
A conaete calculation method of correlation coefficient change rate will hereafter be described. 



25 



(i) Cakrulation of correlation coefficient change rate 



30 



dD(t) 
dt 



» D{t) • max{D(t-l), D(t-2)} 



(8) 



35 The change rate is derived by the difference of correlation coefficient between frames. As a result of 
preliminary experiments, however, it has been found that there is a large dispersion among frames. As 
represented by bk>ck 24-1 of Rg. 10. therefore, a difference is derived with respect to a larger one of 
correlation coefficients of two immediately preceding frames. Of course, a difference may be derived with 
respect to the largest one among 3 frames including a further preceding frame. Considering that the motion 

40 picture image cut typically lasts for at least approximately one second (30 frames), a difference with respect 
to the largest value in the past 0.5 second (15 frames) may k>e defined as the change rate. It does not 
become a txirden on the main memory 16 to store always approximately IS conrelation coefficients 0. In 
case the picture image change in a motion picture image cut is intermittent with a period of several frames, 
an ordinary difference produces a large change rate. When the present invention is used, however, the 

45 value of change rate does not become large because the largest value of correlation coefficients is derived 
by further going back to past frames. 

Fig. 1 1 is a flow chart of image change point registration processing. 

Bk)ck 26 of Pig. 2 is Index information producing processing of a motion picture image cut perfonmed 
when a change point between cuts is determined. Fig. 1 1 shows its detailed operation. 

50 First of all. the variable N is updated by 1 In block 26-1. In block 26-2, the value of variable TM storing 
the current time code is stored into arrangement TIME (N). Thereafter, an image Q1 preceding the change 
point between cuts by one is stored into an index image storage area PE in black 26-3. A current image Q 
is stored into an index image storage area PS in block 26-4. As a result, the start frame and the final alarm 
of a motion picture image cut are obtained as compressed Images for indexing. 

55 Fig. 12 is a diagram showing data structures of various arrangements included in the main memory of 
Rg. 1. 

Fig. 12 shows a memory area 16 capable of storing 100 compressed images, which is divided into 
small areas. i.e.. a time code memory arrangement TIME (100 images) 16-1, an area PS (100, 80, 60) 16-2 
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for storing index inaages of compressed image memory, an area PE (100, 80. 60) 16-3 for storing Index 
images of compressed image memory, compressed image temporary storage memory Q1 (80. 60) 16-4, a 
correlation coefficient time series memory D (3) 16-5, a histogram memory HIST (2. 48. 64) 16-7. and a 
various variables memory 16-8. 
5 As shown in Rg. 12. the present embodiment is so defined that up to 100 compressed images can be 
stacked in the main memory 16 as index images. When the number N of index images exceeds 100, 
however, they may be stored Into the magnetic disk device 17 of the external memory. 
Rg. 13 is a flow chart of frame update processing. 

Rrst of all, variable t1 is substituted into variable t2 in block 27-1. Succeedingly, variable t is substituted 
10 into variable tl . As a result, subscripts for address reference of various arrangements indicating the current 
frame, a frame preceding the current frame by one frame, and a frame preceding the current frame by two 
frames are shifted. Without performing actual data transfer, the completely same effect as that of the case 
where various time series data are transferred is obtained. The image Q is then transferred to an image 
storage area Ql preceding by one. 
15 Rgs. 14A and 14B are time charts showing the change of correlation coefficient and correlation 
coefficient change rate with time obtained when the present embodiment is used. 

As shown in Rg. 14A, the frame correlation coefficient 30 moves to the right as the No. is increased 
from frame No. 0. The central part having large values is a part having large correlation coefficients, i.e., a 
collection of images moving violentiy. All of them are not change points between motion picture image cuts. 
20 On the other hand, the correlation coefficient change rate 31 assumes values ranging from 1 to -1 around 
0 as shown in Rg. 148. Only when this correlation coefficient change rate 31 exceeds 9 fixed threshold 32. 
it is determined that a change point between motion picture image cuts has appeared. 

When a conventional method of setting a threshold for the correlation coefficient 30 is used, false 
detection cannot be avoided in a frame having violent motions located at the central part of Rg. 14. When a 
25 method taking the change rate of Rg. 148 into consideration according to the present invention is used, 
however, false detection can be avoided. 

In this way. the present invention brings atx)ut ttie following effects. 

(a) Image decimation and compression of color picture element values are performed in preprocessing. 
As. compared with input images, therefore, information is compressed to one 256-th. As a result, the 

30 amount of computation for detecting change points is reduced. Without using special hardware, a change 
point between motion picture image cuts can be detected at a practical speed even in an ordinary wori( 

station. 

(b) The correlation coefficient is derived by dividing the screen into 48 small t>locks, finding changed 
blocks on the basis of the degree of resemblance of color histogram between blocks, and dividing the 

35 sum total of these blocks which have changed by the number of all blocks. Unlike the method using 
histogram, therefore, kMvering in detection sensitivity due to missing of information conceming an area is 

obviated. 

(c) In order to finally determine an image change point which is a change point between motion picture 
image cuts, it is decided whether a difference value between a correlation coefficient and the largest 

40 correlation coefficient over several past frames has exceeded a fixed threshold. Even if image changes 
continue or are intermittent with short periods in the same motion picture image cut, therefore, false 
detection of an image change point is prevented. 

As heretofore described, change points of motion picture images can be detected accurately without 
lowering the detection sensitivity and they can be detected at high speed even In an ordinary woric station. 
45 The time code of change point information and compressed images thus derived can be effectively used as 
browsing infonmation for automatically searching heads in vWeo editing and rapidly referring to the whole 
video medium. 

Claims 

so 

1. A method for detecting change points between cuts from motion picture images including a plurality of 
consecutive images, said nnethod comprising the steps of: 

inputting (21) motion picture Images in a time series by taking a frame as the unit, said motion 
picture images being the subject of detection of change points between cuts; 
55 producing (22) a predetermined feature quantity including a cotor histogram possessed by image 

data of said motion pknure images while taking a frame as the unit on the basis of said inputted motion 
picture images; 

producing (23) a correlation coefficient conceming feature quantity between said produced feature 
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quantity of a current frame and a feature quantity produced with respect to a frame immediately 
preceding the cunrent frame: 

producing (24) a change rate between said correlation coefficient and a conelation coefficient 
produced with respect to a frame preceding the current frame; and 
5 detecting (25) a time point as a change point between cuts of said motion picture images, said 

change rate exceeding a predetennined allowable value at said time point. 

2. A motion picture image change point detecting method according to Claim 1, wherein said feature 
quantity includes luminance or color histogram of each of blocks obtained by dividing an image of each 

TO frame into a predetemnined number of blocks. 

3. A motion picture image change point detecting method according to Claim 1, wherein said correlation 
coefficient is derived by dividing (23*8) an image of each frame into a predetermined number of blocks, 
comparing said feature quantities between corresponding blocks of temporally consecutive preceding 

15 and succeeding frames, counting (23-9) blocks in which the result of said comparison has indicated 
such a change as to exceed a predetermined value, and defining (23-11) the number of blocks thus 
counted nonmalized by the number of blocks of the whole image Included in one frame as said 
correlation coefficient. 

20 4- A motion picture image change point detecting method according to Claim 1, wherein said change rate 
is generated from a difference between a correlation value derived for the current frame and the largest 
one of conrelation values derived with respect to a plurality of frames preceding said cunrent frame. 

S. A motton picture image change point detecting method according to Claim 4, wherein said plurality of 
26 frames are arbitrarily selected out of frames ranging from said current frame to a frame preceding in 
time said current frame by 15 frames. 

8. A motion picture image change point detecting method according to Claim 1, wherein when said 
change point t>etween cuts is detected, a time code identifying a frame detected as said change point 
30 between cuts and an Image of a frame of said detected change point between cuts are generated (26) 
as index information of a cut of motion picture images. 
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0 There is provided a method for detecting change 
points between cuts from motion picture images 
including a plurality of consecutive images. First of 
all. in this method, motion picture images which are 
the subject of detection of change point between 
cuts are inputted (21) in a time series t>y taking a 
frame as the unit, and a predetermined feature quan- 
tity including a color histogram possessed by image 
data of said motion picture Images is produced (22) 
while taking a frame as the unit. A correlation coeffi- 
cient with respect to feature quantity between the 
above descrit^ feature quantity and a feature quan- 
tity produced in an immediately preceding frame is 
then produced (23). Further^ a change rate between 
ti)6 conrelation coefficient of tiie cunrent frame and a 
correlation coefficient produced for a frame preced- 
ing the current frame is produced (24). A time point 
whereat the change rate exceeds a predetenmined 
allowable value is detected (25) as a change point 
between cuts of the motion picture images. 
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